Regional weather forecasting is a critical problem for localized climate adaptation, disaster mitigation, and sustainable development. While machine learning has shown impressive progress in global weather forecasting, regional forecasting remains comparatively underexplored. Existing efforts often use different datasets and experimental setups, limiting fair comparison and reproducibility. We introduce IndiaWeatherBench, a comprehensive benchmark for data-driven regional weather forecasting focused on the Indian subcontinent. IndiaWeatherBench provides a curated dataset built from high-resolution regional reanalysis products, along with a suite of deterministic and probabilistic metrics to facilitate consistent training and evaluation. To establish strong baselines, we implement and evaluate a range of models across diverse architectures, including UNets, Transformers, and Graph-based networks, as well as different boundary conditioning strategies and training objectives. While focused on India, IndiaWeatherBench is easily extensible to other geographic regions. We open-source all raw and preprocessed datasets, model implementations, and evaluation pipelines to promote accessibility and future development. We hope IndiaWeatherBench will serve as a foundation for advancing regional weather forecasting research. Code is available at https://github.com/tung-nd/IndiaWeatherBench.

Weather forecasting has attracted increasing attention as climate change brought more heat waves, tropical cyclones, heavy rainfalls and other extreme weather events, affecting millions of people worldwide [1] [2]. Researchers have devised sophisticated numerical schemes and equations to capture complex weather dynamics to improve forecast accuracy[3]. However, its immense computational complexity often necessitates the use of large scale compute clusters, incurring notable energy cost and long processing time, making ensemble forecasts, critical for predicting such events[4], expensive or infeasible. Trained on European Centre for Medium-range Weather Forecast (ECMWF)'s reanalysis product ECMWF Reanalysis v5 (ERA5) dataset, deep learning based weather prediction (DLWP) models have shown promising performance, with FourCastNet[5] being the first data-driven model that directly competes with ECMWF's Integrate Forecasting System (IFS), followed by a series of other models[6][7][8][9] that outperforms IFS in various metrics [10], these models demonstrates excellent ability in capturing global weather trends, with a fraction of the computation requirement.